8-9 t/s并没有完全发挥8060S的潜力，llama.cpp的llama-server有一个小问题导致server配speculative decoding时性能欠佳（与具体硬件无关）：https://github.com/ggml-org/llama.cpp/issues/12968
动手简单修一下这个问题之后Qwen 2.5 72B iq4_xs + 1.5B draft在acceptance rate理想时可达到10-12 t/s左右
https://github.com/hjc4869/llama.cpp/commit/0b32f64ffbe973e99e0dc7097be31d4d966d476e
BY David's random thoughts
The super secure UAE-based Telegram messenger service, developed by Russian-born software icon Pavel Durov, is looking to raise $1bn through a bond placement to a limited number of investors from Russia, Europe, Asia and the Middle East, the Kommersant daily reported citing unnamed sources on February 18, 2021.The issue reportedly comprises exchange bonds that could be converted into equity in the messaging service that is currently 100% owned by Durov and his brother Nikolai.Kommersant reports that the price of the conversion would be at a 10% discount to a potential IPO should it happen within five years.The minimum bond placement is said to be set at $50mn, but could be lowered to $10mn. Five-year bonds could carry an annual coupon of 7-8%.
Telegram has made it easier for its users to communicate, as it has introduced a feature that allows more than 200,000 users in a group chat. However, if the users in a group chat move past 200,000, it changes into "Broadcast Group", but the feature comes with a restriction. Groups with close to 200k members can be converted to a Broadcast Group that allows unlimited members. Only admins can post in Broadcast Groups, but everyone can read along and participate in group Voice Chats," Telegram added.
David& 39;s random thoughts from us